Read the Catch2 dialect, so the guard covers a DuckDB consumer - #126
Merged
estebanzimanyi merged 1 commit intoSep 3, 2026
Merged
Conversation
WITNESS: a MobilityDuck CI run writes its whole test summary as one line of
the Catch2 console reporter DuckDB's test runner builds, reading
... (2695 assertions in 102 test cases)
The rules read surefire, pytest, Go and VSTest, so that log names no dialect
and reaches the no-summary failure: a sqllogictest consumer cannot use the action at all.
MEASURED over that job log: the rules read `catch2`, total 102, 0 skipped, and
a floor of 102 is accepted while 103 is refused. The passing form's test-case
count EXCLUDES the skipped cases -- the reporter writes
`testCases.passed - skippedTests` -- so `3 skipped tests, ... in 99 test cases`
is a suite of 102 with 3 skipped, and the two are added rather than read off
the printed count. The failure table and the wholly-skipped form are read too.
tests/test_test_outcome.py goes 9 cases to 16, and the suite floor 293 to 300.
The order is load-bearing and one of those cases holds it: the failure table's
`101 passed` satisfies the pytest pattern, so Catch2 is read BEFORE pytest --
pytest-first reports 101 as the total and misses the failure and every skip
beside it.
WHY, and what this dialect does NOT do: under this runner a sqllogictest FILE
is one test case, and a `mode skip` inside a file silences the statements after
it while the file still reports as a passing test case. The skip count
therefore cannot see a `mode skip`, and only the floor holds -- it moves when a
whole file leaves the suite. The module says so where a reader meets the
dialect, because a guard whose green answer is silence about the very construct
a consumer uses is worse than none. A sqllogictest consumer needs a census of
`mode skip` in the test sources beside this.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WITNESS: a MobilityDuck CI run writes its whole test summary as one line of
the Catch2 console reporter DuckDB's test runner builds, reading
The rules read surefire, pytest, Go and VSTest, so that log names no dialect
and reaches the no-summary failure: a sqllogictest consumer cannot use the action at all.
MEASURED over that job log: the rules read
catch2, total 102, 0 skipped, anda floor of 102 is accepted while 103 is refused. The passing form's test-case
count EXCLUDES the skipped cases -- the reporter writes
testCases.passed - skippedTests-- so3 skipped tests, ... in 99 test casesis a suite of 102 with 3 skipped, and the two are added rather than read off
the printed count. The failure table and the wholly-skipped form are read too.
tests/test_test_outcome.py goes 9 cases to 16, and the suite floor 293 to 300.
The order is load-bearing and one of those cases holds it: the failure table's
101 passedsatisfies the pytest pattern, so Catch2 is read BEFORE pytest --pytest-first reports 101 as the total and misses the failure and every skip
beside it.
WHY, and what this dialect does NOT do: under this runner a sqllogictest FILE
is one test case, and a
mode skipinside a file silences the statements afterit while the file still reports as a passing test case. The skip count
therefore cannot see a
mode skip, and only the floor holds -- it moves when awhole file leaves the suite. The module says so where a reader meets the
dialect, because a guard whose green answer is silence about the very construct
a consumer uses is worse than none. A sqllogictest consumer needs a census of
mode skipin the test sources beside this.